OCCASIONAL PAPER 



NCVER 



Disability and learning 
outcomes: How much does 
the disability really matter? 



Tom Karmel 
Nhi Nguyen 

National Centre for Vocational 
Education Research 



The views and opinions expressed in this document are those of NCVER and do 
not necessarily reflect the views of the Australian Government or state and 
territory governments. 



Publisher’s note 



To find other material of interest, search VOCED (the UNESCO/NCVER international database 
<http:/ /www.voced.edu.au>) using the following keywords: disability; disadvantaged; educational 
background; educational level; educational needs; outcome of education; training; vocational education 



© Australian Government, 2008 

This work has been produced by the National Centre for Vocational Education Research (NCVER) 
as a joint initiative of the Australian Government and state and territory governments, with funding 
provided through the Australian Department of Education, Employment and Workplace Relations. 
Apart from any use permitted under the Copyright Act 1968 , no part of this publication may be 
reproduced by any process without written permission. Requests should be made to NCVER. 



The views and opinions expressed in this document are those of NCVER and do not necessarily 
reflect the views of the Australian Government or state and territory governments. 

ISBN 978 1 921412 56 1 web edition 

TD/TNC 93.01 

Published by NCVER 
ABN 87 007 967 311 

Level 1 1, 33 King William Street, Adelaide, SA 5000 
PO Box 8288 Station Arcade, Adelaide SA 5000, Australia 

ph +61 8 8230 8400 fax +61 8 8212 3436 
email ncver@ncver.edu.au 
<http:/ / www.ncver.edu.au> 

<http:/ / www.ncver.edu.au/ publications/ 2017.html> 




About the research 




NCVER 



Disability and learning outcomes : Horn much does the disability really 
matter? by Tom Karmel and Nhi Nguyen, NCVER 



In 2005, the National Centre for Vocational Education Research (NCVER) produced a 
statistical compendium examining vocational education and training (VET) students with a 
disability as a whole group; it also compared different disability groups, focusing on their 
participation levels, achievements and outcomes from VET in 2003 (Cavallaro et al. 2005). 

The report found that, on the whole, educational achievements and outcomes from VET are 
relatively poor for students reporting a disability, but there is considerable variability between 
types of disability. 

In addition, educational achievement prior to commencing VET was found to have some 
bearing on students’ results in VET. Generally, students reporting a disability in VET have 
lower prior education and poorer outcomes from VET than all other VET students. This 
implies that the poor educational performance of students reporting a disability may be due to 
their educationally disadvantaged position rather than their disability. 

This paper, Disability and leaning outcomes: How much does the disability really matter ? by Tom Karmel 
and Nhi Nguyen, seeks to highlight the direct effect of the disability by controlling for the 
background characteristics (notably educational background, but also age, level of study and 
field of study) in a simple statistical model. 



Key messages 

V The study finds that with disabilities such as hearing/ deaf, intellectual, acquired brain 
impairment and vision, the actual disability explains little, once we take into account other 
student characteristics such as age, sex, educational background and course studied. 

V By contrast, both student characteristics and the disability itself directly impact on the low 
completion rates of those with a physical or mental illness or a medical condition. 

Overall, the significant point to emerge is that it is not helpful to treat students with a disability as 
one group. The different disability groups have students with different background characteristics, 
and the direct effect of the disability on academic performance differs between groups. 



Tom Karmel 

Managing Director, NCVER 
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Introduction 



In 2005, the National Centre for Vocational Education Research (NCVER) produced a statistical 
compendium examining vocational education and training (VET) students with a disability as a 
whole group; the compendium also compared different disability groups, focusing on their 
participation levels, achievements and outcomes from VET in 2003 (Cavallaro et al. 2005). 

The report found that, on the whole, educational achievements and outcomes from VET are 
relatively poor for students reporting a disability, but there is considerable variability between types 
of disability. 

In addition, educational achievement prior to commencing VET was found to have some bearing 
on students’ results in VET. Generally, students reporting a disability in VET have lower prior 
education and poorer outcomes from VET than all other VET students. This implies that the poor 
educational performance of students reporting a disability may be due to their educationally 
disadvantaged position rather than their disability. 

Cavallaro et al. (2005) acknowledged the need for a better understanding of how prior education 
impacts on learning outcomes for students reporting a disability. This paper seeks to throw light on 
the relative importance of educational background for students reporting a disability and on their 
educational outcomes. 1 



Background 

In 2003, there were some 91 400 students reporting a disability, representing around 5.3% of all 
students in the NCVER students and courses collection. Around 20% of the general population 
reports some form of disability (ABS 2003). As can be seen from table 1, the largest group 
consisted of students reporting a physical disability. The figure shows a fair spread of students 
across the other disability categories. Students reporting a disability have generally low educational 
attainment levels, with almost half having only completed Year 10 or lower. 



1 We measure this by using module completion rate. The module completion rate is calculated as a proportion of 

successful completion of a VET module/ subject (Pass + Recognition of prior learning + Non-assessable satisfactorily 
completed divided by Pass + Fail + Withdraw + RPL + NA satisfactorily completed and NA not satisfactorily 
completed). These calculations are based on the population derived for this analysis and may differ slightly from the 
published figures in the disability compendium. 
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Table 1 Students by disability, 2003 



Disability type 


No. reporting a 
disability 


% 


% with Yr 10 
or lower 


Hearing/deaf 


10 593 


11.53 


50.3 


Physical 


18 149 


19.75 


49.8 


Intellectual 


8 871 


9.65 


45.9 


Learning 


12 352 


13.44 


59.9 


Mental illness 


7 668 


8.34 


47.7 


Acquired brain impairment (ABI) 


1 858 


2.02 


46.4 


Vision 


13 778 


14.99 


44.0 


Medical condition 


15 394 


16.75 


48.8 


Other 


12 878 


14.01 


48.8 


Unspecified 


7 008 


7.63 


28.9 


Total students with disabilities 


91 902 a 


100.00 


47.1 



Note: (a) Students can have more than one disability type; therefore, the base used for the total number of students with 

disabilities is the total number reporting a disability. 

Source: National VET Provider Collection, unpublished data 2008. 



Table 2 shows the relative module completion rates for students by disability type. Overall, the 
module completion rate is almost eight percentage points lower than for students not reporting a 
disability. However, there is considerable variation by type of disability, with students with a mental 
illness having the poorest module outcomes, with a module completion rate of 63.5% compared 
with 80.2% for students not reporting a disability. Students with hearing or visual disability, two of 
the better performing groups, had module completion rates almost five percentage points lower 
than that for students without a disability. 

Table 2 Module completion rates for students reporting a disability, by disability type, 2003 



Disability type 


Module 

completion rate 
(%) 


Flearing/deaf 


75.4 


Physical 


72.1 


Intellectual 


74.3 


Learning 


69.4 


Mental illness 


63.5 


Acquired brain impairment 
(ABI) 


72.1 


Vision 


75.1 


Medical condition 


70.0 


Other 


69.5 


Unspecified 


81.0 


All students reporting a 
disability 


72.5 


Other students* 3 ’ 


80.2 



Note: (a) Figure includes students for whom disability status is 'not known’. 

Source: National VET Provider Collection, 2003 unpublished data. 



Now we also know that educational background is related to the module completion rate. As can 
be seen from table 3, those with post-school qualifications have a higher module completion rate 
than students who have only completed Year 12 or lower. 
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Table 3 Module completion rates by highest educational background for all students, 2003 



Educational attainment 
level 


Module 

completion rate 
(%) 


Bachelor degree and above 


81.5 


Advanced diploma 


79.5 


Diploma 


80.3 


AQF Certificate IV 


80.7 


AQF Certificate III 


80.9 


AQF Certificate II 


77.8 


AQF Certificate 1 


77.7 


Year 12 


76.0 


Year 1 1 


77.8 


Year 10 


76.7 


Year 9 or lower 


71.7 


Other 


80.3 


Unknown 


85.9 


No prior education 


71.0 



Source: National VET Provider Collection, 2003 unpublished data. 



We know that students with a disability tend to have relatively poor educational backgrounds, so 
the question is how important is disability in addition to educational background. 

The simple approach we take to isolate the effect of disability as well as educational background is 
to run a simple additive regression model, in which disability and educational background are 
entered in the model separately. We also control for other characteristics: qualification being 
undertaken, age, field of education and gender. 



Findings 

The model results are shown in appendix A. As expected there is a relationship between 
educational background and the module completion rate. Those students with no prior education 
are least likely to successfully complete modules, followed by students with only up to Year 9 
schooling. At the other end of the spectrum those with degrees, diplomas and certificate IV 
perform the best. 

However, our primary interest is the disability variables. All the parameter estimates for the 
disability categories are negative, with the exception of the intellectual disability category. The 
easiest way to understand the results is to report two sets of module completion rates, and this is 
done in table 4. From table 2 we have the original module completion rate (column 1, from 
table 2). From the model we can calculate the adjusted module completion rate. This is the rate for 
each disability type, assuming that the students in the disability group have the same characteristics 
(apart from the possession of a disability) as the whole student population. The direct effect of the 
disability is given by column 3 (the coefficients of the disability dummy variables in table Al). 
Finally, the difference between the adjusted rate and the original module completion rate gives the 
indirect impact of the disability (column 4, from table 4); that is, the effect of the student 
characteristics of the disability group in question. 
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Table 4 Students by disability and module completion rates (%), 2003 



Disability type 


Original module 
completion rate 


Adjusted module 
completion rate 


Direct effect of 
disability on 
module 

completion rate 
(percentage 
points) (b) 


Effect of student 
characteristics on 
module 

completion rate 
(percentage 
points)* 3 ’ 


Hearing/deaf 


75.4 


78.2 


-1.7 


-2.8 


Physical 


72.1 


76.4 


-3.5 


-4.4 


Intellectual 


74.3 


81.2 


1.2 


-6.9 


Learning 


69.4 


77.9 


-2.1 


-8.5 


Mental illness 


63.5 


70.2 


-9.8 


-6.7 


Acquired brain 
impairment (ABI) 


72.1 


79.1 


-0.8 


-7.0 


Vision 


75.1 


79.5 


-0.4 


-4.4 


Medical condition 


70.0 


75.0 


-5.0 


-5.0 


Other 


69.5 


75.2 


-4.8 


-5.7 


Unspecified 


81.0 


80.3 


0.3 


0.7 


Other students* 3 ’ 


80.2 


79.9 




0.2 



Notes: (a) Figure includes students for whom disability flag is ‘not known'. 



(b) Difference between the adjusted module completion rate for a particular disability and the overall adjusted 
completion rate for students not reporting a disability (80.0%). 

(c) Column 1 minus column 2. 

Source: National VET Provider Collection, 2003 unpublished data. 

To aid explanation, take the hearing/ deaf disability category. The original module completion rate for 
this group is 75.4%, and this compares with an overall module completion rate of 80.0%. If we assume 
that the hearing/ deaf disability students have the same characteristics as the whole student population 
control (with respect to age, previous education, level of qualification being undertaken, field of study), 
then the completion rate for hearing/ deaf disability students would be 78.2%. So we see that the 
hearing/ deaf disability group has characteristics other than their disability that are associated with 
relatively low completion rates. The difference between the original and the adjusted rates 
(-2.8 percentage points) represents the effect of the student characteristics of the hearing/ deaf 
disability group on their completion rate. The direct effect of their being in the hearing/ deaf disability 
group is taken from the relevant coefficient in the regression (-0.017 or -1.7 percentage points). 

Another way of thinking about these estimates is to observe that the difference between the 
original module completion rate and the overall completion rate can be split into the direct effect 
of the disability and the effect of the student characteristics of the group; that is, 75.4%-80.0% = 

-4.6% = -1.7%-2.8% 2 . Those readers interested in a formal derivation are directed to appendix C. 

Returning to the results shown in table 4. Essentially, these results imply that we can divide the 
disability types into two categories. 

The first applies to the types of disability for which the explanation behind poor completion rates 
lies in the (non-disability) characteristics of the student and what they are studying, rather than their 
disability. These students include: 
hearing/ deaf 
intellectual 
learning 

acquired brain impairment 
•y" vision 

•y" unspecified disability. 



2 The discrepancy is due to rounding. 
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It must be qualified that this interpretation does not exclude the effect of a student’s disability on 
their poor completion rates. The point here is that it is the non-disability characteristics that are 
directly attributable to poor completion rates for students in the above disability groups. The 
disability itself may affect these characteristics — such as low prior educational attainment levels — 
but it is not the disability itself that direcdy affects poor completion rates in vocational education 
and training. 

The other category applies to the types of disability for which poor completion rates can be 
attributed to the disability, over and above the effect of the other characteristics of the affected 
students. By contrast with the first group, poor performance of students in the second category can 
be traced back to both the direct impact of the disability and the impact of the other characteristics, 
and they include: 

"y" physical 

-v* mental illness 

•y" medical condition 

•y" other disability'. 



Discussion 

So what are the implications of these findings? Importantly, it must be acknowledged that the 
analysis we have undertaken is at an aggregate level and it does not necessarily assist in 
understanding the support needs of individual students. However, what it does show is that in 
commenting on the poor performance of students with a disability, and in designing systems to 
help these groups, we need to recognise a number of points. 

First, there is significant variation in educational performance between types of disability, and 
therefore it is not helpful to treat students with a disability as one group. 

Second, for some types of disability' — hearing/ deaf, intellectual, learning, acquired brain 
impairment, vision, unspecified disability' — the actual disability explains little in a statistical sense 
once we take into account the other characteristics of the students (age, sex, educational 
background, course studied). This suggests that the learning support offered to students in these 
categories to address their special requirements is doing a good job. Perhaps additional support 
offered should be addressing the impacts of non-disability characteristics and, as such, be the same 
as that offered to students in general who tend to struggle because of these factors (for example, 
early school leavers). 

Finally, for other types of disability' the actual disability does appear to directly contribute to poor 
educational performance (in addition to other characteristics which by themselves lead to poor 
performance). Thus this group of students — those with a physical disability or mental illness — 
would appear to need special assistance relating to this disability. It is interesting to see those 
reporting a ‘physical’ disability included in this second category. A physical disability — much like 
hearing/ deaf, intellectual, learning, and vision — is recognisable and more easily identifiable than 
something like mental illness. As such, support for these students can be more easily catered for. 
On the other hand, perhaps a physical disability or mental illness chiefly affects attendance and 
drop-out rates, resulting in poor completion of courses and subjects (Miller & Nguyen 2008). 

Overall, the main point to emerge is that it is not helpful to treat students with a disability as one 
group. The different disability groups have students with different background characteristics, and 
the direct effect of the disability on academic performance differs between groups. 
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Appendix A: Regression results 



Data 

Data used in this analysis are from the National VET Provider Collection on students undertaking 
study in 2003. The 2003 ‘publication scope’ was used to categorise all disability groups for this 



analysis and this should be noted for any future comparison. Only table 1 in the paper 
current (2006) scope to define disability groups. 

Table A1 Regression results dependent variable is the module completion rate 


uses the 


Variable 


DF 


Parameter 

estimate 


Standard 

error 


t Value 


Pr > |t| 


Intercept 


1 


0.816 


0.009 


88.0 


<.0001 


Prior education level 












Bachelor or above 


1 


0.096 


0.009 


10.5 


<.0001 


Advanced diploma 


1 


0.078 


0.009 


8.2 


<.0001 


Diploma 


1 


0.096 


0.009 


10.4 


<.0001 


Certificate IV 


1 


0.115 


0.009 


12.4 


<.0001 


Certificate III 


1 


0.093 


0.009 


10.1 


<.0001 


Certificate II 


1 


0.088 


0.009 


9.5 


<.0001 


Certificate 1 


1 


0.078 


0.009 


8.3 


<.0001 


Year 12 


1 


0.063 


0.009 


6.8 


<.0001 


Year 1 1 


1 


0.064 


0.009 


7.0 


<.0001 


Year 10 


1 


0.048 


0.009 


5.3 


<.0001 


Year 9 or lower 


1 


0.012 


0.009 


1.3 


0.194 


Unspecified 


1 


0.079 


0.009 


8.6 


<.0001 


Unknown 


1 


0.088 


0.009 


9.7 


<.0001 


No prior education 


- 










Disability 












Hearing/deaf 


1 


-0.017 


0.003 


-5.1 


<.0001 


Physical 


1 


-0.035 


0.003 


-13.4 


<.0001 


Intellectual 


1 


0.012 


0.004 


3.3 


0.001 


Learning 


1 


-0.021 


0.003 


-6.5 


<.0001 


Mental illness 


1 


-0.098 


0.004 


-24.6 


<.0001 


Acquired brain impairment (ABI) 


1 


-0.008 


0.008 


-1.0 


0.325 


Vision 


1 


-0.004 


0.003 


-1.4 


0.175 


Medical condition 


1 


-0.050 


0.003 


-17.6 


<.0001 


Other 


1 


-0.048 


0.003 


-15.5 


<.0001 


Unspecified 


1 


0.003 


0.004 


0.8 


0.424 


No disability 


- 
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Variable 


DF 


Parameter 


Standard 


t Value 


Pr > |t| 






estimate 


error 






Major qualification level undertaken 
Diploma and above 


1 


-0.224 


0.001 


-179.6 


<.0001 


Certificate IV 


1 


-0.254 


0.001 


-206.5 


<.0001 


Certificate III 


1 


-0.172 


0.001 


-156.9 


<.0001 


Certificate II 


1 


-0.189 


0.001 


-162.3 


<.0001 


Certificate 1 


1 


-0.229 


0.002 


-150.8 


<.0001 


Other education 


1 


-0.084 


0.001 


-77.7 


<.0001 


Non-award courses 


- 










Age groups 












Under 15 


1 


-0.023 


0.004 


-6.3 


<.0001 


Age 15-19 


1 


-0.043 


0.001 


-30.9 


<.0001 


Age 20-24 


1 


-0.044 


0.001 


-31.1 


<.0001 


Age 25-44 


1 


-0.038 


0.001 


-29.0 


<.0001 


Age 45-64 


1 


-0.011 


0.001 


-7.9 


<.0001 


Field of education 












Natural and physical sciences 


1 


0.077 


0.004 


17.4 


<.0001 


Information technology 


1 


0.037 


0.002 


23.2 


<.0001 


Engineering and related technologies 


1 


0.149 


0.001 


125.3 


<.0001 


Architecture and building 


1 


0.148 


0.001 


101.9 


<.0001 


Agriculture, environmental and related 
studies 


1 


0.135 


0.002 


87.9 


<.0001 


Health 


1 


0.152 


0.001 


107.1 


<.0001 


Education 


1 


0.152 


0.002 


86.2 


<.0001 


Management and commerce 


1 


0.077 


0.001 


70.2 


<.0001 


Society and culture 


1 


0.079 


0.001 


63.3 


<.0001 


Creative arts 


1 


0.081 


0.002 


47.0 


<.0001 


Food, hospitality and personal services 


1 


0.151 


0.001 


119.7 


<.0001 


Mixed field programs 


- 










Subject only 


1 


0.086 


0.002 


52.8 


<.0001 


Male 


1 


-0.011 


0.001 


-18.5 


<.0001 


Linear hypothesis test: Prior educational attainment has no effect on the module completion rate 


Source 


DF 


Mean 

square 


F value 


Pr > F 




Numerator 


13 


64.510 


547.70 


<.0001 




Denominator 


1.71E6 


0.118 








Linear hypothesis test: Disability has no effect on the module completion rate 






Source 


DF 


Mean 

square 


F value 


Pr > F 




Numerator 


10 


20.230 


171.75 


<.0001 




Denominator 


1.71E6 


0.118 
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Appendix B 



Table B1 Average values of independent variables in calculation of adjusted module completion rate 



Variable 


Average value (%) 


Highest education level 




Bachelor and above 


5.13 


Year 10 


14.18 


Year 1 1 


7.13 


Year 12 


18.83 


Year 9 


6.56 


Advanced diploma 


1.36 


Diploma 


2.51 


Certificate IV 


2.42 


Certificate III 


5.65 


Certificate II 


2.76 


Certificate 1 


1.18 


Unspecified 


4.93 


Unknown 


27.27 


Qualification undertaken in 2003 




Diploma and above 


11.01 


Certificate IV 


11.57 


Certificate III 


23.35 


Certificate II 


15.39 


Certificate 1 


5.06 


Other education 


18.64 


Age group 




Under 15 years 


0.57 


15-19 years 


20.79 


20-24 years 


16.15 


25—44 years 


38.54 


45-64 years 


18.93 


Field of education 


0.00 


Natural and physical sciences 


0.37 


Information technology 


4.22 


Engineering and related technologies 


15.21 


Architecture and building 


5.87 


Agriculture, environmental and related studies 


4.81 


Health 


5.98 


Education 


3.38 


Management and commerce 


21.43 


Society and culture 


10.68 


Creative arts 


3.40 


Food, hospitality and personal services 


9.26 


Subject only 


5.78 


Male 


51.21 



Note: These figures are based on the population used in the regression analysis and, due to missing values being excluded 

from the analysis, these proportions may not match the publication figures. 

Source: Based on figures from the National VET Provider Collection, 2003 unpublished data. 
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Appendix C 



Decomposing module completion rates 



Our interest is in working out what component of a disability group’s module completion rate can 
be attributed to the characteristics of the group and what component can be ascribed to the 
disability. The approach is to use a standard decomposition, as follows. 

Let £[>’,] be the expected value of the completion rate for disability group i, and E\y _ be the 
expected value of the completion rate for all students. 

Then dr,] = CL i + /3 'x i where OL i is the coefficient of the dummy for the disability group i, X ( . is 
a vector of characteristics (age, highest level of education, level of study, field of study) for the 
members of disability group i, and /3 is the vector of the corresponding coefficients. 

Similarly, £[v]- p X where X is a vector of characteristics (age, highest level of education, level 
of study, field of study) for the members of the whole student population. 



The direct effect of the disability is given by CL j . 

The adjusted completion rate for disability group i is given by^[v,-]=a i ,+p'x. 



The indirect effect of the disability, that is, the effect of the characteristics of the members of the 
disability group, is given by /3 (x, — x). 

So we see that E\y t ]- f?[}’]= a i + j3 'x, - (Ex = a t . + /3 ' (x, - x). 
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